Skip to content

feat(mothership): v1.0.0 - #8208

Merged
waleedlatif1 merged 308 commits into
stagingfrom
feat/mothership-revamp
Sep 23, 2026
Merged

waleedlatif1 merged 308 commits into
stagingfrom
feat/mothership-revamp

Conversation

@Sg312

@Sg312 Sg312 commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Mship v1

Type of Change

  • New feature

Testing

Manual

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…m exploration run 5

Lint: trigger-category blocks are entry blocks (schedule was an orphan); references inside Function code are checked with the runtime tokenizer; the agent-cli lint and grep read the draft state instead of the sanitized export. Catalog: operation inputs publish sub-block ids only (no canonical-param aliases); trigger-category blocks expose their trigger-mode fields. Tables: dispatch processedCount counts unlimited dispatches; json-language code fields accept objects. Files: restore keeps the folder when it still exists. Sandbox tools: outputTable failures report the files already written and the per-language result shape; the file writer keeps the computed result.
…r restore, dispatch listing, group column attach, import block summary; logs query and deps engines; run-tool errors and payload compaction; stale tests aligned

Editing: router routes validated as {id?, title, value} with unknown keys named; malformed connections reported instead of dropped; sub-block value() defaults evaluated on add so a webhook gets its token. Tables: folder restore no longer 500s (lock inside the row transaction), completed dispatches are listed, groups attach to existing output columns. Workflows: import answers with its blocks. Mothership engines: logs query resolves bare paths under output, marks missing paths, drops non-executed runs under --where; deps lists graph predecessors and child return shapes; run-from-block validation errors reach the agent; run payloads compact input.code. CLI generated types regenerated.
…eps mock skeleton; run tools lift the terminal output and take select; lint all-clear names what it checked; docs chunker keeps code spans
…ys it removes the output columns and their data; a publish that lands on public auth prints a note
…erence paths against output schemas; logs stats segments; requiredWhen and model ids in the agent catalog; live tag usage; one row-filter grammar; minted service-account credential ids; webhook delivery URLs on deployment status; mv semantics for folder moves

Executor: <block.error> resolves empty on a block that succeeded so a shared error collector is writable; a number input given a non-numeric string fails the run at start naming the field; a regression test pins that failure traces carry child spans. Lint: a reference whose first path segment is not on the block's effective outputs (or its responseFormat schema) is an unknown-field finding. API: logs stats honours segmentCount and omits empty buckets unless includeEmpty; block detail publishes requiredWhen instead of flattening conditional requirements and always lists model options with hosted marks; knowledge tag usage counts through the document slot; every table rows filter accepts a bare condition; service-account credentials mint their id; deployment status lists each webhook's delivery URL; a folder move into an existing folder moves it inside. OpenAPI and CLI generated types regenerated.
…c required-field lint; folder moves to the root; empty-graph and no-entry lint notes; span durationMs; MCP tools reported on undeploy and listed inactive; paged connector types and workspaces; grep --in refuses unknown selectors; run_function names exported files; run tools surface the failing block's error

Export takes includeWorkspaceBindings for same-workspace round trips and import answers with warnings naming every block whose required binding was stripped. Lint reports a missing required field for every block type (the knowledge base id was skipped), notes an empty graph and a graph with no entry block, and the trace spans carry durationMs. Undeploy answers with the MCP tools it archived and the tools list shows them inactive. connector-types list is paged (25, summary projection, detail=full) and workspaces list defaults to 25. The agent CLI's grep refuses a prefix or unknown --in selector with the accepted forms; a run_function that exported files and wrote a table says both; a run tool whose executor result carries no message uses the failing block's own error.
…thout one; column rename reports unmigrated Table-block filters; lint checks table fields against the live schema; runCount counts every settled run; runs cancel is honest about no-ops; dry-run apply answers with previewBlockIds; conditionResult is the chosen test's boolean with selectedTitle; workflows run --select-output keys blockOutputs by the selector and rejects unknown heads; knowledge search exposes rankScore and rank; the multi-trigger error names workflows state get; the Knowledge block declares cost, tokens, model; media writes refuse to fork a moved folder; grep --in knowledge points at knowledge search
…rns the group's run state and outputs; knowledge upload prints the document fields; docs chunks titled by page not nav link; logs carry hasHandledErrors and opt-in handledErrorRuns; outputTable receipt keeps stdout
…red without a format is written as image/jpeg bytes, not as its base64 text under the json format (thumbnails opened as raw text)
… image references (never listings or writes); an unresolved reference image fails the call instead of rendering without it; copilot session-sandbox calls are priced like Function-block sandboxes and report the raw cost beside the billed one; CLI docs regenerated
A bare --in selector (e.g. --in agent, --in file_v5) materialized every searched
world to find one block: 65 block details plus every workflow state per call. On
dev that took 18-34s per grep and tripped the per-user rate limit. Each world now
has a cheap index (its listing) and a per-resource fetch; a --in search reads the
indexes and fetches only the matches. Whole-world searches are unchanged.
…face

The copilot back-derives fixes from the strings sim hands it, so a retired name
becomes a wrong instruction to the user (dev 2026-09-03: a save_upload mention
became "drag the photo into the files panel"). Every model-facing string audited
today now names the current surface:

- upload notice: a chat upload lives at uploads/<name> and is not a workspace
  file; workflows import takes --workflow (there is no --file); a .zip is mounted
  and unzipped in the sandbox rather than a files unzip path that does not resolve
- table import resolves uploads/<name> directly (includeChatUploads) instead of
  pointing at the retired save_upload tool
- function-execute / generate-image: outputs get, files ls, files restore and
  tables list replace read/grep/glob/restore_resource and Go VFS meta.json paths
- process-contents: browser/terminal pointers no longer name browser_* or a
  terminal tool this surface does not have; docs fallback names docs search
- lint/deps usage strings use the plural workflows group
- integration credential error names credentials list
The copilot answers users from these pages (docs search), so a wrong sentence
becomes a wrong instruction. Audit fixes:

- chat: uploads live under uploads/, not the Files panel; on-demand workspace
  reach instead of a per-message snapshot; Chat runs Opus 4.8; connectors and
  deletes/restores as they actually behave; login pages via the shared browser
- cli: --select-output works on sync runs (only --async conflicts); runs get
  takes the same block selectors; secrets set needs --scope; workflows run runs
  the deployment or --manual; sim profiles honours --output; which groups have
  no singular alias; the embedded CLI has no profile, login, or config; @path
  reads the chat sandbox
- model defaults are claude-sonnet-5 (guardrails has none); API examples use
  https://www.sim.ai/api/v2 (apex 301s POSTs into GETs)
- quick reference / shortcuts: only affordances that exist (no Deploy tab, no
  workspace duplicate/export, Mod+B not Mod+E, variables under the ⋯ menu,
  <variable.name> syntax)
- editor read-only rules match round-trip-safety.ts; card links fixed
- sim-cli: runs-get selector error and the Settings label match the product;
  batch delete describes its ids as deleted, not updated; docs/api regenerated
The workspace-file handler has always supported operation=create with a
new_file target, but the generated tool schema (from the copilot catalog)
forbade both, so the watched write could never create a file: a "save this
report as a workspace file" turn died on `/operation must be equal to one of
the allowed values` (dev 2026-09-03). Regenerated from the catalog that now
declares create, new_file, and fileName; regression test on the validator.
…h platform catalogs

The embedded CLI and the agent-cli engines were typed v2 clients pointed at the
server's own URL: every tool call was a network round trip through the proxy,
API-key auth, the abuse rate limits, and the proxy body ceiling. A grep over one
block definition cost 8-34s and tripped the per-key limit (dev 2026-09-03).

- sim-cli: ResolvedProfile / EmbeddedCliIdentity take an optional transport; the
  client sends through it instead of fetch. The installed CLI never sets it.
- sim: an in-process transport resolves a v2 path against a generated route table
  (scripts/generate-v2-route-table.ts, check:v2-route-table in check:audits) and
  invokes the route handler directly. The request is marked internal through a
  WeakSet — not a header — so admission still authenticates it but skips the
  pre-auth IP bucket and the per-key rate limits, which exist for callers on the
  wire. Contracts, use cases, presenters, and error envelopes are the ones the
  network path runs. Anything outside the v2 table falls through to fetch.
- grep engine: blocks and tools (5,000 built-ins at 100 a page) are memoized per
  workspace; an exact block or tool id (--in file_v5) resolves from that corpus
  without listing any workspace world; name fragments still index every world
  and fetch only the matches.
…the dev hang is understood

Every cli_grep on dev timed out at 60s from the first deploy of 99aad28. The
transport, marker, route table, and grep memo stay; the embedded identity goes
back to the HTTP path while the hang is diagnosed from sim's logs.
…he file text by version

The first dev turn on the in-process transport fired eight world-wide greps at
once. Each rebuilt every world concurrently on the one process serving the chat
— 24 block-catalog listings, 150 tool-catalog pages, 130 file reads in sixty
seconds — until all eight timed out. Over the wire the same fan-out had been
spread across tasks and throttled by the network.

- concurrent greps now await the build already in flight for a (world, workspace)
- nested requests are bounded to eight for the whole process, not per world
- platform corpora (blocks, tools) are kept an hour, not ten minutes
- a file's text is cached by id + updatedAt + size, so repeat greps re-read only
  files that changed
The engine read only -C; -A and -B were ignored silently, so `-A 40` returned
the bare match line and the model concluded context flags were unreliable
(dev 2026-09-03). A bad value is refused like -C's.
Under skipValidation the value arrives as the raw string it was deployed with;
the sandbox lease's finite check rejected it, so on dev every run_code failed
as "Boot sandbox" (Sandbox pricing multiplier must be a finite nonnegative
number). getCostMultiplier now goes through envNumber, as the env module's own
rule says numeric overrides must.
The agent spent nine tool rounds and ~20K tokens learning what exists in the
workspace before an orientation task could start. The chat request now carries
a compact inventory (contracts ChatRequest.inventory): workflows, tables,
knowledge bases, files, skills, custom tools, MCP servers, credentials, and
secret names — names and ids, one page per world, capped worlds named in
`truncated`. It is read through the same use cases the v2 listings run, under
the caller's session principal, so authorization is unchanged; a world that
fails to list is left empty and logged rather than failing the turn. The worker
renders it once per turn as a request-local message.
…aller's principal

The tools world paged 5,000 built-in tools through the route stack at 100 a
page — fifty round trips, each re-resolving the gate and walking the registry —
which was the whole cold cost of a world-wide grep. The embedded bridge now
resolves the delegation key to its principal exactly as the v2 surface does and
hands it to the engines; with it, the tools world reads the catalog through
listCatalogTools in one call — same authorization, same projection. Without a
principal the client path stays.
…1 §6)

Sim side of copilot background tasks. copilot_task_subscriptions
(migration 0315) records a worker task's watch on a workflow execution;
the logging session's completion attempt posts every subscribed run's
outcome to the worker. POST /api/mothership/tasks/subscribe and
POST /api/mothership/wake (internal key): the wake runs the inbox's
headless lifecycle under the task id as message id, announces itself on
the chat status channel so an open chat reconnects, persists the user
message with origin 'task', and resolves the pill in the arming turn.

Client: a 'task' content block end to end (run handler, turn-model node,
both serializer directions, persisted normalizer — which also flattened
plan blocks to text on reload —, display block, segment, TaskPill) and a
muted TaskNotificationRow for task-origin messages. The stream
validator's run kinds now derive from the generated contract; the
generated mothership-stream-v1 gains task_armed / task_delivered.
@waleedlatif1

Copy link
Copy Markdown
Collaborator

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

Comment thread apps/desktop/src/main/browser-agent/cdp.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2334 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2334 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2334 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2334 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

waleedlatif1 added a commit that referenced this pull request Sep 23, 2026
PR #8208 widened Google Calendar's domain-wide delegation scopes to include
calendar.readonly for every service-account token. Google refuses the whole
token exchange when any requested scope is not authorized for the client, so
indexed admin-mode Calendar syncs configured for calendar.events.readonly
failed every run and were eventually disabled.

Live service search only needed the wider scope to read an all-day event's
calendar time zone. events.list returns the calendar's timeZone under
calendar.events.readonly, so live search now reads it there and runs on the
same single scope as indexing; save-time validation therefore covers live
sources too.

The admin calendar picker impersonates with calendar.calendarlist.readonly and
resolves names from the admin's calendar list. When Google refuses a delegated
token, the picker, save-time validation, and live search share one message
that names the scopes an administrator must authorize, and a delegated 401/403
in service mode no longer tells the member to reconnect their own account.
@waleedlatif1
waleedlatif1 merged commit f734ba5 into staging Sep 23, 2026
69 checks passed
@waleedlatif1
waleedlatif1 deleted the feat/mothership-revamp branch September 23, 2026 23:43
waleedlatif1 added a commit that referenced this pull request Sep 23, 2026
PR #8208 widened Google Calendar's domain-wide delegation scopes to include
calendar.readonly for every service-account token. Google refuses the whole
token exchange when any requested scope is not authorized for the client, so
indexed admin-mode Calendar syncs configured for calendar.events.readonly
failed every run and were eventually disabled.

Live service search only needed the wider scope to read an all-day event's
calendar time zone. events.list returns the calendar's timeZone under
calendar.events.readonly, so live search now reads it there and delegates with
the same single scope as the indexed crawl, matching Drive and Gmail. The admin
calendar picker keeps its own calendar.readonly request, as documented.
waleedlatif1 added a commit that referenced this pull request Sep 24, 2026
* fix(execution): resolve stored file references only for workspace members

Workflow file inputs can reference stored files by id, key, or internal URL,
and every resolved file is added to the run's explicit file grants. Any
caller could use this, so an anonymous public-API, public MCP, chat, or
webhook caller could pull another workflow's run files or workspace files
into a run and receive a presigned URL for them.

Execution now derives a stored-file reference scope from the run's
principal: authorized member principals (session, personal/workspace API
key, OAuth token, delegated) resolve workspace-wide; system principals may
only reference files already stored under the current execution, which keeps
chat and webhook uploads working. The default is the restricted scope, so a
new caller fails closed. Input-format defaults are workflow-authored and
still resolve workspace-wide.

Outside workspace scope, an upload whose URL is an internal file URL is
resolved as the stored reference it names rather than downloaded with the
run user's access, and a key from another execution is refused before it is
looked up, so the refusal does not reveal whether the file exists.

* fix(mothership): withhold unregistered table secrets from sim_cli results

Table rows read through sim_cli reached the model without their persisted
secret provenance, so stored secrets in cells were never redacted. Row use
cases now report the provenance of the rows they return to an observing
transport (mirroring the workspace-file delivery observer), and the agent CLI
table transport imports it into the tool call's registry, answers 503 without
a registry, and marks the registry incomplete when a row-bearing table route
returns without reporting provenance. Provenance reported by detached work
after the call settles is ignored.

Export download links are refused outright: a signed link to the whole table
as plaintext CSV cannot carry provenance once fetched.

Run-state and enrichment error text (runState.error, blockErrors, enrichment
provider errors) is captured from executor output without its secret
provenance, so a read that returns any of it is withheld as well; reads whose
run state carries no error text are unaffected.

* fix(search): run live Calendar service search on the crawl scope

PR #8208 widened Google Calendar's domain-wide delegation scopes to include
calendar.readonly for every service-account token. Google refuses the whole
token exchange when any requested scope is not authorized for the client, so
indexed admin-mode Calendar syncs configured for calendar.events.readonly
failed every run and were eventually disabled.

Live service search only needed the wider scope to read an all-day event's
calendar time zone. events.list returns the calendar's timeZone under
calendar.events.readonly, so live search now reads it there and delegates with
the same single scope as the indexed crawl, matching Drive and Gmail. The admin
calendar picker keeps its own calendar.readonly request, as documented.

* fix(search): keep results painted when Show more widens the page

* fix(chat): use the defined brand token for the resource activity dot

* fix(knowledge): document connector auth in the v2 contract and reject $NAME secrets

* fix(mothership): resolve Copilot env-reference passwords for chat deploy and file share

Copilot now deploys chats and shares files through the v2 API, which stored a
whole-value {{NAME}} password literally (or 400'd a short one on length). The
application use cases now resolve the reference from the effective environment
when, and only when, the caller is an admitted Copilot workspace invocation and
the password will actually be stored (password mode; for a file share, only
while enabling it), refuse an unset variable by name, and hold the resolved
value to the password rules. Every other principal keeps literal semantics.

The v2 password fields admit a whole-value reference below the password
minimum as one refined string (not a union, which would make the CLI flag
JSON-only), still capped at the password maximum; the use cases enforce the
15-character rule on the value actually stored. Connector API-key references
share the same principal environment lookup, and the exact reference regex now
lives in one module.

Unlike the removed Copilot tool path, the resolved password is not recorded in
a resolved-secret trace registry: the v2 use case has none, and the password
is write-only and never echoed back.

This branch was previously deployed

1 inactive deployment
Preview — 60436b0b Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants